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ABSTRACT 

A major problem encountered by both teachers and 
authors of instructional materials is the sequencing of instructional 
activities. A review of the literature is given and it is pointed out 
that there is substantial evidence to support the general theory of 
the hierarchical structure of knowledge. A learning hierarchy for the 
computational skills for addition of rational numbers with like 
denominators was constructed. Based on the hypothesized ordering of 
the subordinate levels, a test was constructed to assess mastery at 
each level in the hierarchy; it was administered to 163 elementary • 
school children in grades 4 through 6. The pass-fail relationships 
were analyzed using the following indirect validation procedures 
adaptable for use with test data: item difficulty, the AAAS approach, 
the Guttman technique, pattern analysis, and correlation analysis. 
Materials consisted of an 11-lesson programed booklet on the addition 
of rational numbers with like denominators. Fourth grade Ss were 
selected to participate in the study on the basis of their 
performance on two pretests. The hierarchical orderings of the 11 
subtasks generated by each of the indirect validation procedures are 
given in tabular form. A pattern analysis technique was used to 
determine the index of agreement of each ordering with the expected 
patterns. This index was above .86 for all orderings except the 
textbook and random sequences. Neither planned nor post hoc 
comparisons showed any significant differences between the logical 
sequence group and the other sequence groups on achievement, 
transfer, or retention. Careful analyses of instructional objectives 
to reveal the prerequisite subtasks is an adequate procedure for 
developing a valid hiererchy. (CK) 
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A major pioblem encountered by both teachers and authors 
of Instructional materials is the sequencing of instructional 
activities (Bartung, 1969s Hickey and Newton 1964; Gagne, 1967; 
Briggs, 1968; Heimer, 1969)* Gagne (1963) stated that M the 
design of an instructional situation is basically a matter of 
designing a sequence of topics” • Do optimal learning sequences 
exist? How are they determined, and how are they verified? 

RELATED LITERATURE 

There is substantial evidence to support the general theory 
of the hierarchical structure of knowledge* Gagne and Brown, 1961; 
Gagne and Paradise, 1961; Gagne, Mayor, Gars tens and Paradise, 

1962; Gagne 1962; Gagne 1963; Gagne and staff. University of 
Maryland Mathematics Project, 1965 have demonstrated that new skills 
and knowledge emerge from lower order knowledge, and that there is a 
significant amount of positive transfer from each successive subordi- 
nate level to the next higher level in a hierarchical ordering of 
such levels* 



The research reported herein was performed pursuant to a grant 
with the Office of Education, U.S* Department of Health, Education 
and Welfare. Purdue Research Foundation, Grant No. QEG-5-70-0020 (509) . 

The research was conducted while Professor Phillips was a research 
assistant at Purdue Educational Research Center, Purdue University 
Lafayette, Indiana. 
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Evidence suggests that optimal learning sequences exist* 
Recent studies of sequencing (Niedermeyer, drown & Sulzen, 1969; 
Brown, 1970) Indicated that Ss using materials sequenced according 
to learning hierarchies performed reliably better than Ss using 
materials Whose sequence was scrambled relative to time to 
complete the instructional program, errors made on the program, 
and on a criterion test of complex problem solving skills* Brown 
(1970) concluded that when a sequence involves tasks that are 
complex problem solving behaviors ordering is an important 
factor in learning even for bright and relatively mature learners* 
In summarizing research on sequencing mathematical tasks Miller 
(1969) concluded that mastery of Individual subtasks In a 
hierarchy can be achieved in several ways, 1' eluding learning 
from randomly ordered sequences but that logical sequencing 
appeared best In terms of overall efficiency and effectiveness* 
Several studies (Roe, Case, and Roe, 1962; Levin and Baker, 1963; 
Payne, Krathwohl, and Gordon, 1967; Ililler, 1965) suggest that 
varying sequences of instructional stimuli which have high 
interdependency does not make much difference in effectiveness 
of instruction* However, some of these studies are plagued with 
design problems. Before sequencing instructional materials in 
mathematics for use in classrooms the effects of sequence upon 
time to achieve the terminal behavior, achievement, transfer, and 
retention should be investigated* 

That sequence is a critical variable in learning has been 
suggested by many learning theorists (Ausubel, 1963; Bruner, 1964: 
Gagne, 1965; Glaser, 1964; and Suppes, 1966). However, both 
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Gagne (1968) and Pyatte (1969) have pointed out that 
determining the hierarchical ordering of sub tasks from simplest 
to most complex is still a major problem. Gagne's (1962) approach 
to learning hierarchy validation by means of trial and error Is 
tedious and costly. While its validity is not questioned, it 
Is unlikely that classroom teachers or authors of Instructional 
materials will validate hierarchies in such a manner. If instru- 
ctional materials are to be improved by hierarchical analyses , a 
less expensive procedure for validating hierarchies is needed. 

The Guttman technique (Torgerson, 1958), pattern analysis 
(Rlmoldi & Grib, 1960) and the AAAS method (AAAS Commission on 
Science Education, 1968) appear to be adaptable for validating 
deductively analyzed hierarchies using test data. Other procedures 
based on the correlation between test items or levels of the hierarchy 
and the difficulty of items could be useful in validating a hierarchy 
using test data. The present study was designed to examine the 
adequacy of each of these procedures for validating a learning hier- 
archy using test data by sequencing instructional materials according 
to the hierarchy generated by each procedure comparing achievement , 
transfer, retention and time to complete the instructional sequences. 

METHOD 

Development and Analyses of the Hierarchy 

Using Gagne's task analysis a learning hierarchy for the compu- 
tational skills for addition of rational numbers with like denominators 
was constructed. The sequence of subtasks generated was reviewed by 
four authors of elementary mathematics texts. Based upon their 
evaluation of the adequacy and completeness of the hierarchy, the 
sequence of subtasks was revised until there were no obvious flaws in 
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the learning hierarchy. Based on the hypothesized ordering of the 
subordinate levels » a test was constructed to assess mastery at each 
level in the hierarchy. The test was designed to minimize chance or 
careless errors. A procedure of test construction similar to the 
"H-technique" (Stouffer, Borgatta, Hays, and Henry, 1952) was adopted. 
The test consisted of composite test items for each level in the hier- 
archy. Each composite item consisted of three items testing the same 
subordinate task. Pass at each level was defined as correct responses 
to at least two of the three items for that level. The entire test 
consisted of 11 composite items making a total of 33 Items. The 
internal consistency of the test was determined using the Kuder- 
Richardson Formula 20 (Nunnally, 1967). 

The test was administered to 163 elementary school children in 
grades 4 through 6 to obtain a sufficiently wide range of ability 
levels. The test was administered by the classroom teachers and 
was completed by all Ss in one sitting although it was not timed. 

Ss were instructed to attempt all items and were given sufficient time 
to do so. 

The pass-fail relationships were analyzed using the foilwing 
indirect validation procedures adaptable for use with test data.* 

(1) Item difficulty (Nunnally, 1967) 

(2) The AAAS approach (AAAS Commission on Science Education, 1968) 

(3) The Guttman technique (Torgerson, 1958) 

(4) Pattern analysis (Rlmoldl and Grib, 1960) 

(5) Correlation analysis (Phillips, 1971) 

For comparative purposes the 11 subtasks were also ordered according to 
the "usual" textbook sequence and foria control they were randomly 
ordered. Thus, 7 different orderings of the 11 sub tasks were generated. 

1. For detailed description of these validation techniques, see 
Phillips, 1971. 
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Instructional Materials 



The materials used in this study consisted of an eleven-lesson 
programmed booklet on the addition of rational numbers with like 
denominators. The program utilized one lesson for each of the 11 
levels in the learning hierarchy. Each of the 11 lessons was designed 
to develop the specific skill represented by the corresponding hier- 
archy level. The lessons were from 2 to 3 pages in lengthy making 
a total program length of 29 pages. The program evoked frequent 
responses from S which he wrote in blanks provided. Care was taken 
to minimize the reading load. While lessons were presented in seven 
different orders from the exterior all seven forms of the instruction 
booklet appeared the same. 

Experimental Procedures 

Fourth grade Ss were selected to participate in the study on 
the basis of their performance on two pretests. Pretest I was designed 
to determine if the learners had mastered the necessary prerequisites 
for successfully achieving the skills presented in the programmed text. 
The test concentrated on the concept of fraction 9 recognizing parts of 
a whole , reading and writing fractional numerals , whole number additions 
and simple whole number division. Pretest I was administered to 175 
Ss one week prior to initiation of the learning sequence. Pretest II 
was administered only to those students judged* on the basis of Pretest I* 
ready to undertake the programmed materials. 

Pretest II was designed to determine if the students had already 
mastered the skills to be taught in the instructional sequence. The 
test consisted of one item for each of the 11 levels in the hierarchy. 

- 5 - 
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Only those subjects judged, on the basis of Pretest II, to have mastered 
an Insignificant number of the skills in the instructional program were 
included in the study. 

Ss were assigned randomly to treatment groups. They worked 
through the programmed booklets independently devoting approximately 
30 minutes per day to the materials until they were completed. £ explained 
how to use the materials and assisted Ss with any problems they encountered 
for the first two days of the study. Thereafter the teachers supervised 
Ss' work until completion of the study. 

Ss entered their responses to questions directly In the booklets 
u3ing a cardboard cover-up for the answer column. Subjects were Instructed 
to keep the answer column covered until they entered their response then 
pull the cardboard down to reveal Immediately the correct response. If 
their response was incorrect and they could not determine the source 
of their error, they were instructed to ask the teacher for help. Teachers 
were instructed to give help only in the context of each child's material. 
For Instance, if one child's sequence had a frame which involved writing 
the simplest name for fractional numerals but no frame preceding dealt 
with the definition of simplest name or the mechanics of renaming, the 
teacher did not show the student the manipulations involved in renaming. 

The students were to be guided in using only the information and art 
work provided in the given frame. 

The teachers kept a log book of the exact number of minutes spent 
on the booklets each day. As each child finished his booklet, the date 
and time of completion were entered in his booklet. Thus allowing for 
determining the total number of minutes each student spent in completing 
the programmed materials. In order to discourage Ss from rushing through 
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the materials, they were rerlnded at the beginning of each session that 
they were to study the materials and try to remember what they did, not 
just copy in the correct responses. They were also told that they would 
be tested upon completion of the booklet. For the duration of the study, 

Ss had no mathematics class other than the work session on the programmed 
booklets. Since they finished the booklet at differing times, teachers 
had other unrelated mathematics activities for each Ss to do until the 
whole group had finished. 

An achievement test designed to measure acquisition of the terminal 
task - addition of rational numbers with like denominators -was administered 
on the day following coup let ion of the programmed sequence. The test 
consisted of two items per level in the hierarchy (lessons in the 
programmed materials) except for two levels which pertained to renaming. 

An alternate form of this test was administered two weeks later as a retention 
test. During these two weeks Ss studied mathematical topics other than 
operations with fractions. A transfer test on subtraction of rational 
numbers with like denominators was administered on the day following 
administration of the achievement test. This test consisted of 10 
items of skills analogous to those found in the learning program on 
rational number addition. Reliability coefficients for all tests were 
determined by the Kuder-Rlchardson Formula 20 (ftunnally, 1967). Analysis 
of variance for multiple groups, unequal n's model, (Winer, 1962) was 
used to investigate the differential effects of sequencing on four 
variables: achievement, transfer, retention and time (total number of 

minutes to complete the programmed booklet.) 




RESULTS OF THE STUDY 



Hierarchical Analyses 

The hierarchical orderings of the 11 subtasks generated by each of 
the indirect validation procedures are given in Table 1* A pattern 
analysis technique (Rlmoldl and Grib, 1960) was used to determine the 

Insert Table 1 about here 

index of agreement of each ordering with the expected patterns. This 
index was above .86 for all orderings except the textbook and random 
sequences. These were .62 and .61 respectively. 

Pretests 

Ss were selected on the basis of their performance on two pretests. 
Pretest I was designed to answer the question "Does S_ have the necessary 

prerequisite skills needed to master the skills presented in the learning 
program?" Pretest II was designed to answer the question "How many of 
the skills presented in the learning program has the learner already 
mastered?" An acceptable score on Pretest I was defined as one ranging 
between 24 and 17 on a 24 point test; for Pretest II between 0 and 4 on 
an 11 point test. Of the 175 Ss tested, 142 met the criterion on both 
tests. Eighty-seven percent of the subjects Included in the study 
obtained scores on Pretest I of 20 or higher with 23% having perfect 
scores. Only 13% of the subjects gave incorrect responses to 5 or 
more items. The mean score on Pretest I was 21.84. Seventy-one of 
the subjects tested were unable to respond correctly to any of the 
11 items on Pretest 11. Ninety-two percent of the subjects gave 
correct responses to less than 3 of the 11 items. The mean score on 
Pretest II was 0.563. 
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The Effects of Sequence 



The differential effects of sequence upon achievement » transfer, 
retention, and time to complete the program were investigated using an 
analysis of variance design. The internal consistency coefficients (KR-20) 
for each test used was greater than .90. One-way analyses of variance 
on achievement, transfer, retention, and time are shown in Tables 2, 3, 4, 5, 
respectively • 

Insert Tables 2, 3, 4, 5 about here 

No overall significant differences were found at the .05 level. However, 
the F-ratio of 2.12 for the analysis of variance on retention was very 
near the critical value 2.15. Post-hoc comparisons of all means using 
the Duncan Multiple Range Test (Winer , 1962) showed statistically significant 
differences at the .05 level between the AMS sequence group (8.52) and 
both the textbook (4.95) and the item difficulty (5.37) sequence groups. 

Further Investigations of Sequence Effects 

Examination of the tests revealed that many low scoring Ss had 
actually mastered the skills Involved in rational number addition. However, 
due to not following directions or having not mastered the skills involved 
in renaming they failed to write answers in simplest form. In view of this 
two other scoring procedures were used. The first of these alternative 
scorings gave one-half credit for responses which were correct but not 
reduced to lowest terms. The second alternative gave full credit for 
responses which were not .In lowest terms but otherwise correct. One-way 
analyses of variance on achievement using the alternative scoring procedures 
are shown in Tables 6 and 7. No overall significant differences were 
found in either case. 

Insert Tables 6,7 about here 
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One-way analysis of variance on transfer allowing partial credit 
In scoring Is shown In Table 8. The differences among treatment means 
was not significant at the .05 level. One-way analysis of variance on 
transfer disregarding reduction to lowest terms in scoring is shown In 
Table 9. The F-ratlo was significant at the .05 level. The Duncan 
Multiple Range Test Indicated significant differences at the .05 level 
between two pairs of means , that of the random sequence group (5.26) and 
both the phi coefficient sequence group (5.10) and the textbook sequence 
group (5.19). 

One-way analyses of variance on retention using the alternative 
scoring procedures are shown in Tables 10 and 11 respectively. In the 
first case F was near the critical value at the .05 level. The Duncan 
Multiple Range Test indicated significant differences between two pairs 
of means. The AAAS sequence group mean was significantly greater than 
that of both the item difficulty and the textbook sequence groups. When 

Insert Tables 8,9,10,11 about here 

disregarding reduction to lowest terms, the F-ratio was significant at the 
•05 level. Significant differences between two pairs of &?ans were found 
using the Duncan test. The AAAS sequence group mean was significantly 
greater than those of both the item difficulty and the textbook sequence 
groups . 

DISCUSSION AND CONCLUSIONS 

The reader's attention was directed toward two troublesome problems 
with many studies of the effects of sequence reported in the literature. 
First, in comparing the effects of a logical and a rrclrn sequence upon 
learning, it was not demonstrated that Indeed a logic,.! sequence and an 
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unbiased random sequence were being used. Second, in many of the studies 
reported, it was suspect el that too many of the subjects already knew much 
of the material presented in the learning program. This study was designed 
to minimize the possibility of these pitfalls. 

The index of agreement was used to determine if the hypothesized 
ordering developed through the use of task analysis was indeed logical. 

That is, that it was hierarchical in structure. The index of agreement 
was .87 which indicated that the observed response patterns of the subjects 
correlated highly with the expected patterns indicating that the logical 
ordering was logical. The index of agreement for the random order was 
.61. Thus, the logical ordering appeared to have markedly more of the 
characteristic of hierarchical structure than did the random ordering. 

With the excpption of the textbook ordering all other sequeces were 
validated empirically using various procedures. The indices of agreement 
for all validated orderings were above .85 indicating high correlations 
between observed and expected response patterns. The index of agreement 
for the textbook ordering was only .62 however. 

Subjects Included in the study had to meet stringent criteria on 
two pretexts. Namely, they had to have the necessary prerequisites for 
undertaking s!*udy of the skills presented in the instructional program, 
and they could not have already mastered the skills to be taught. Thus, 
the probability was very low that outcomes attributed to sequence were 
affected by inadequate entering behaviors or by prior knowledge of 
the material to be learned. 

On the other hand due consideration must be given to two sources 
of artifact over which we had less control than would have been desireable. 

1. Teachers were instructed on the type end amount of help 

to provide. However, they reported that Ss in some sequence 
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groups who were asked to perform certain tasks when they 
had not mastered necessary prerequisites were very frustrated 
In these instances* the teachers may have provided too much 
instruction making assessment of sequential effects difficult 
This could have accounted for the absence of significant 
differences among the mean achievement scores of the seven 
sequence groups. 

2. Examination of Ss responses revealed that many students 

did not write answers in lowest terms. Again* the teachers 
were Instructed to stress directions and be sure all learners 
understood what was expected of them. Thus* it might be con- 
cluded that the lessons pertaining to reducing to lowest 
terms were not adequate. However* when allowing partial 
credit or disregarding reducing to lowest terms in scoring* 
still no significant differences were found on immediate 
achievement. 

Helther planned nor post hoc comparisons showed any significant 
differences between the logical sequence group and the other sequence 
groups on achievement, transfer* or retention. The logical sequence 
group did require significantly less time to complete the program than 
did the correlational sequence group. This suggests that careful task 
analysis of instructional objectives can be a powerful tool in devising 
optimal instructional sequences. In fact it may mean that in terms of 
overall cost* that careful analyses of instructional objectives to 
reveal the prerequisite subtasks is an adequate procedure for developing 
a valid hierarchy. 
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Within the limitations of this study the results seem to justify 
the following conclusions: 

1. The overall efficiency of the learning process, using programmed 
instructional materials, can be affected by changing the 
sequential ordering of the sub tasks. 

2. Sequence, even if random, has little effect upon immediate 
achievement. 

3. Retention appears to be the variable, of the four under study, 
most suscep table to sequence manipulation. 

4. No sequence maximally facilitated achievement, retention, and 
transfer, and required less time to complete. However, based 
on the group means, the AAAS procedure yielded the best 
sequence overall. 

5. Textbook authors may need to give more careful confederation 
to the sequencing of subtasks within major topics or sub* 
divisions of a chapter. 

6. Optimal Instructional sequences can be derived using learning 
hierarchies validated from test data. 

The effects of sequence should be Investigated by replications with 
more complex skills involving longer learning sequences and larger samples. 
Further research should attempt to determine the effects of sequence upon 
attitudes and anxieties experienced by learners in different sequence 
groups, the interaction effects between sequence and ability, and the 
effects of sequence upon immediate achievement, transfer, and long 
term retention. The effects of carefully sequenced Instructional 
materials according to validated learning hierarchies on the performance 
of the slow learner and the remediational value of such instructional 
sequences should be investigated thoroughly. 
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Tablo 2 i Ono-v/ay analysis of varianco on achievement* 



Source of 
Variation 


KS 


df 




F 


P 


Treatments 

Experimental 

error 


18.83 

12.93 


<5 

135 




1.45 


.1992 




Group Means 








1 2 


3 


4 


5 


5 


7 


(5.23 4.C4 


5.47 


5.21 


5.05 


5.14 


7.52 



Tablo 3 * Onc-v.*ay analysis of varianoo on transfer# 



Source of TS df F 

Variation 



Treat'. ;cnts 11 #30 6 1*^3 

Experimental 7*08 135 

error 

rr mr~> MiwNj't rum* " *11 *r i i ♦! 



*12051 




—»«■ rn^tm 



Croin> rc-ciiU) 



4 

3.95 



5 



3.5? 



1 

5.05 



2 

4,53 



3 

5.25 



3.5? 5. 84 



Table A i Onc-v/ay analysis of varianco on retention* 



Source of 
Variation 


ICS 


df 




F 


P 


Treatments 

Experimental 

error 


33.61 

15.84 


6 

135 




2.12 


.0542 




Group moans 








1 2 


3 


4 


5 


6 


% 

7 


7.50 7.32 


6.89 


5.37 


7.62 


4.95 


8.52 






Table 5 • 0ne-\?ay analysis of variance on 


time* 




Source of 
Variation 


r>3 df 


F 


P 


Treatments 

Experimental 

error 


2931.20 6 
1493.99 135 


1.96 


© 

4» 

[ 

1 




Group means 






1 2 


345 


6 


7 


103.86 101 . 9 $ 


114.16 126.11 135.33 


117.24 122.66 
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2aWe 6 . One-way analysis of variance on achievement 
(partial credit). 



Source of 
variation 


ks 


df 


7 


P 


Treatments 


43.21 


6 


1,688 


,1278 


Experimental 

error 


25*60 


135 





Croup means 

1 2 3 4 5 6 7 

7.42 9.89 ' 7.05 8 . 6 ? 7i81 11,05 



tab!.e 7 . One-way analysis of variance on achievement 
(reducing disregarded)* 



Source of 
variation 


183 


df 

'i' • 


F 


P 


Treatments 


71.43 


6 


1.70 


*1259 


Experimental 

error 


42,12 


135 







Croup mcan3 

12 3 4 5 6 7 

11.91 9.68 13.05 8.4? 11,00 11,10 14,05 



Table 8 



» Ono-wav analysis of variance on transfer (partial 
eredit) 



Source of 
Variation 


m 


df 


? 

e 


P 


Treatments 


19.97 


% 

6 


1.89 


•08?0 


Experimental 

error 


10.59 


135 








. Group moans 






1 2 


3 


* 5. 


6 


7 


6.27 5.89 

ft 


6.89 


5.00 4.76 


4.52 

a 

> 


6.86 



Table 9 • One-way analysis of variance on transfer (reducing 
disregarded)* 



Source of 
Variation 


153 


df 


F 


P 

ft 


Treatments 


31.43 


6 


2.1? 


.0465 


Experimental 


14.31 


135 






error 












Group means 






1 2 


3 


4 5 


6 


7 


7,05 6.84 


8.2 6 


5.95 5.10 


5.19 


7.86 



20 

o 

ERIC 



r 



Table 10 . One-way analysis of varlanoe on retention (partial 
eredit). 



Source of 
Variation 


KS 


*f 


F 


F 


Treatments 

Experimental 

error 


56.32 

26.35 


6 

135 


2.1b 


•0525 




Group means 






1 2 


3 


4 5 


6 


7 


10.91 10.26 


9.70 


7*84 10.81 


7.57 


12.10 



Table ll • One-way analysis of variance on retention (reduc- 
ing disregarded)* 



Source of 
variation 


IS 


df P 


P 


Treatments 


95.08 


6 2.25 


.0416 - 


Experimental 

error 


42.24 

e 


135 






Group means 




1 2 


3 


4 5 6 


7 


14.23 12.84 


12.53 


9.63 13.90 10.00 


15.48 





\ 



